OCPBUGS-66219: fix: kubernetes/conformance suite execution from list and failure propagation#82
Merged
openshift-merge-bot[bot] merged 3 commits intoredhat-openshift-ecosystem:mainfrom Dec 5, 2025
Conversation
33c616d to
37f5c1b
Compare
mtulio
added a commit
to mtulio/provider-certification-tool
that referenced
this pull request
Dec 1, 2025
Added init container to dynamically extract Kubernetes conformance tests from the OTE k8s-tests-ext binary to support OCP 4.20+ where the kubernetes/conformance suite was removed from openshift-tests. Changes: - Added extract-k8s-conformance-tests init container - Extracts k8s-tests-ext.gz from hyperkube image - Filters conformance tests using jq or grep as fallback - Saves test list to /tmp/shared/k8s-conformance-tests.list - Tests container uses extracted list via entrypoint-tests.sh - Gracefully falls back to default suite if extraction fails This ensures the kubernetes/conformance suite continues to work on OCP 4.20+ clusters while maintaining backward compatibility with earlier versions. Related: redhat-openshift-ecosystem/provider-certification-plugins#82 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
This commit implements two critical fixes for OCP 4.20+ compatibility: 1. **Failure propagation mechanism** - Detects when blocker plugins fail and stops dependent plugins - Prevents wasted execution time on tests that can't succeed - Exception: artifacts collector (plugin 99) always runs - Uses plugin ID check for cleaner, more maintainable logic - Location: openshift-tests-plugin/pkg/plugin/plugin.go:716 2. **Kubernetes conformance test extraction support** - Consumes extracted test list from init container - Checks for /tmp/shared/k8s-conformance-tests.list - Falls back to default suite if extraction unavailable - Works with init container defined in opct repository - Location: openshift-tests-plugin/plugin/entrypoint-tests.sh:59-76 Failure propagation chain: - 05 (upgrade) → blocks → 10 (kube-conformance) - 10 (kube-conformance) → blocks → 20 (conformance-validated) - 20 (conformance-validated) → blocks → 80 (replay) - 80 (replay) → blocks → 99 (artifacts-collector) - Plugin 99 (artifacts collector) always runs regardless of failures Related: redhat-openshift-ecosystem/opct#183 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
37f5c1b to
da39782
Compare
mtulio
added a commit
to mtulio/provider-certification-tool
that referenced
this pull request
Dec 2, 2025
Added init container to dynamically extract Kubernetes conformance tests from the OTE k8s-tests-ext binary to support OCP 4.20+ where the kubernetes/conformance suite was removed from openshift-tests. Changes: - Added extract-k8s-conformance-tests init container - Extracts k8s-tests-ext.gz from hyperkube image - Filters conformance tests using jq or grep as fallback - Saves test list to /tmp/shared/k8s-conformance-tests.list - Tests container uses extracted list via entrypoint-tests.sh - Gracefully falls back to default suite if extraction fails This ensures the kubernetes/conformance suite continues to work on OCP 4.20+ clusters while maintaining backward compatibility with earlier versions. Related: redhat-openshift-ecosystem/provider-certification-plugins#82 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
Collaborator
|
/lgtm |
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: jcpowermac The full list of commands accepted by this bot can be found here. The pull request process is described here DetailsNeeds approval from an approver in each of these files:
Approvers can indicate their approval by writing |
901d102
into
redhat-openshift-ecosystem:main
13 checks passed
mtulio
added a commit
to mtulio/provider-certification-tool
that referenced
this pull request
Dec 5, 2025
Added init container to dynamically extract Kubernetes conformance tests from the OTE k8s-tests-ext binary to support OCP 4.20+ where the kubernetes/conformance suite was removed from openshift-tests. Changes: - Added extract-k8s-conformance-tests init container - Extracts k8s-tests-ext.gz from hyperkube image - Filters conformance tests using jq or grep as fallback - Saves test list to /tmp/shared/k8s-conformance-tests.list - Tests container uses extracted list via entrypoint-tests.sh - Gracefully falls back to default suite if extraction fails This ensures the kubernetes/conformance suite continues to work on OCP 4.20+ clusters while maintaining backward compatibility with earlier versions. Related: redhat-openshift-ecosystem/provider-certification-plugins#82 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
mtulio
added a commit
to redhat-openshift-ecosystem/opct
that referenced
this pull request
Dec 5, 2025
## Summary This PR addresses OCP 4.20+ compatibility issues where the `kubernetes/conformance` suite was removed from openshift-tests, requiring dynamic extraction from the k8s-tests-ext binary. Additionally, it refactors the client architecture to improve code organization and maintainability. ## Problem Starting with OCP 4.20, the `kubernetes/conformance` suite is no longer available in openshift-tests. This causes the kube-conformance plugin to fail when trying to run conformance tests. Additionally, the client initialization code was scattered across multiple packages, making it harder to maintain and test. ## Solution ### 1. Dynamic Test Extraction (OCP 4.20+ Support) Added an init container (`extract-k8s-conformance-tests`) to the kube-conformance plugin that: 1. Retrieves the hyperkube image from the cluster release 2. Extracts the k8s-tests-ext.gz binary 3. Runs the binary to list all tests and filters conformance tests 4. Saves the test list to `/tmp/shared/k8s-conformance-tests.list` 5. The tests container uses this list via the entrypoint script ### 2. Suite Name Enforcement by Cluster Version Added logic to automatically detect cluster version and set the appropriate suite name: - **OCP 4.19 and earlier**: Uses `kubernetes/conformance` (default) - **OCP 4.20+**: Uses `kubernetes/conformance/parallel` (new parallel suite) Implementation: - Added `setSuiteName()` function in `pkg/run/run.go` that: - Detects cluster version using the ClusterVersion API - Parses version to determine if >= 4.20 - Sets `suiteNameKubernetesConformance` ConfigMap key accordingly - Plugin consumes this via `DEFAULT_SUITE_NAME` environment variable ### 3. Client Architecture Refactoring Consolidated client creation into a unified `Client` struct: **Before**: ```go kclient, sclient, err := client.CreateClients() restConfig, err := client.CreateRestConfig() ``` **After**: ```go cli, err := client.NewClient() // Access via cli.KClient, cli.SClient, cli.RestConfig ``` **Benefits**: - Single source of truth for client initialization - Easier to pass clients between functions - Better testability with comprehensive unit tests - Reduced code duplication **Files Updated**: - `pkg/client/client.go` - New unified Client struct - `pkg/client/client_test.go` - Comprehensive unit tests (24 test scenarios) - `pkg/run/run.go` - Updated to use new client architecture - `pkg/run/validations.go` - Updated function signatures - `pkg/status/status.go` - Updated to use new client - `pkg/destroy/destroy.go` - Updated to use new client - `pkg/cmd/adm/e2ed/taintNode.go` - Updated to use new client ## Changes ### Core Implementation Files **`data/templates/plugins/openshift-kube-conformance.yaml`** - Added `extract-k8s-conformance-tests` init container (73 insertions) - Extracts tests using `jq` with grep fallback for compatibility - Gracefully handles failures by falling back to default suite - Consumes `DEFAULT_SUITE_NAME` from ConfigMap for version-specific suite selection - Fixed yamllint issues (line length, string continuation) **`pkg/run/run.go`** - Added `setSuiteName()` function to detect cluster version and set suite name - Updated to use new unified `Client` struct - Enhanced dry-run mode to print ConfigMaps as JSON - Fixed nil pointer dereference by correcting variable shadowing - Added suite name to plugin configuration via ConfigMap **`pkg/client/client.go`** - Refactored to use unified `Client` struct with KClient, SClient, and RestConfig - Replaced `CreateClients()` with `NewClient()` for better encapsulation - Made `createRestConfig()` private for better encapsulation **`pkg/client/client_test.go`** (NEW) - Added comprehensive unit tests for client creation - 24 test scenarios covering success and error paths - Tests for environment variable priority and configuration fallback ### Supporting Files **`pkg/run/validations.go`** - Updated function signatures to accept RestConfig as parameter - Removed dependency on client package for config creation **`pkg/status/status.go`** - Updated to use new Client struct - Fixed missing assignment bug in client initialization **`pkg/destroy/destroy.go`** - Updated to use `NewClient()` instead of `CreateClients()` **`pkg/cmd/adm/e2ed/taintNode.go`** - Updated to use new client architecture **`.containerignore` / `.gitignore`** - Added patterns for better build artifact management ## Backward Compatibility - ✅ **OCP 4.19 and earlier**: Falls back to `kubernetes/conformance` suite - ✅ **OCP 4.20+**: Uses `kubernetes/conformance/parallel` suite dynamically - ✅ **Graceful degradation**: If version detection or test extraction fails, uses default suite - ✅ **Client refactoring**: Fully backward compatible, no API changes to external consumers ## Related PRs - Plugins PR: redhat-openshift-ecosystem/provider-certification-plugins#82 - Adds entrypoint script support to consume extracted test list - Adds `getSuiteName()` function to support `DEFAULT_SUITE_NAME` environment variable - Adds failure propagation to stop dependent plugins on failure ## Testing ### Validation Performed - ✅ All tests pass: `make test` (30 packages) - ✅ Go vet clean: `make vet` - ✅ YAML linting passes: `yamllint` - ✅ Build successful: `make build` - ✅ Client unit tests: 24 test scenarios covering all code paths - ✅ Dry-run mode tested with verbose output ### CI Validation - Validated with CI rehearsal job on OCP 4.21 cluster - All linter checks passing (go_lint, yaml) ## Migration Notes For developers updating code that uses the client package: **Before**: ```go kclient, sclient, err := client.CreateClients() if err != nil { return err } ``` **After**: ```go cli, err := client.NewClient() if err != nil { return err } // Use cli.KClient, cli.SClient, cli.RestConfig ``` --- 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> --------- Co-authored-by: Claude <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR implements two critical fixes for OCP 4.20+ compatibility and improved CI efficiency:
Changes
1. Failure Propagation (plugin.go)
Problem: When a conformance plugin fails, subsequent dependent plugins continue running and waste execution time on tests that cannot succeed.
Solution: Added failure detection in
RunDependencyWaiter()that:Code change (
openshift-tests-plugin/pkg/plugin/plugin.go:714-719):Failure propagation chain:
2. Kubernetes Conformance Test Extraction (entrypoint-tests.sh)
Problem: OCP 4.20+ removed the
kubernetes/conformancesuite from openshift-tests.Solution: Added logic to consume extracted test list from init container:
/tmp/shared/k8s-conformance-tests.list(created by init container in opct repo)Code change (
openshift-tests-plugin/plugin/entrypoint-tests.sh:59-76):Benefits
Related PRs
Testing
Validated with CI rehearsal jobs on OCP 4.20+ clusters.
🤖 Generated with Claude Code
Co-Authored-By: Claude noreply@anthropic.com